1. Field of the Invention
This invention relates to an imaging system. More particularly this invention relates to a super-resolving imaging apparatus having increased depth of field which can be employed in collision avoidance systems for vehicles, and for distance estimation applications in general.
2. Description of the Related Art
Nowadays distance estimation is mainly performed using two basic techniques: active sensing and image processing. In the active sensing category are included radar-type devices which illuminate the region of interest by scanning and then estimating the distance by measuring the time between the transmitted and received signals. Although such devices are relatively simple, they are presently too expensive to be practical in private vehicles. These devices have limited resolution, and are generally too slow for real-time collision avoidance applications for terrestrial vehicles.
The other major category distance estimation devices are passive systems based on detector arrays, such as CCD, CMOS, and ICCD gated CCD, which employ advanced image processing algorithms. Such systems, in order to obtain algorithmic convergence, require high resolution and thus have a restricted field of view. These systems also suffer from a limited depth of focus. In addition, the postprocessing algorithms that perform the time-to-impact evaluation require heavy computation. Despite their present limitations, the passive systems are constantly being upgraded in technological performance, and are being dramatically reduced in price.
Technical problems in the field of optical scanning are similar to those of the immediate field of the invention. Imaging based scanners are required to convert spatial degrees of freedom in order to obtain high horizontal and vertical resolution, and adequate depth of focus. Here are several related patents that deal with high resolution and extended depth of field scanning of objects as bar codes. Several disclosures are known to deal with similar problems. U.S. Pat. No. 5,657,147 discloses an optical scanner which includes a light source and optical deflector for scanning a light flux from the light source. Beams with extended confinement for scanning purposes, as proposed in U.S. Pat. No. 5,315,095, may be the basis for scanners usable, for example in a bar code reader, by generating a multitude of Gaussian beams derived from a common light source. A scanner using this technique has an extended working range. An optical scanner for reading barcodes detected within a large depth of field is disclosed in U.S. Pat. No. 4,748,316, which proposes the use of a plurality of scan lines having different focal lengths. If at least one of the scan lines is small relative to irregularities of the surface on which the label appears, an extended depth of scanning can be achieved. U.S. Pat. No. 4,820,911, describes an apparatus which is adapted to be hand-held and utilizes anamorphic optics for scanning and reading a bar code. The optics provides a beam having an oblong cross-section in the nominal plane of the bar code. U.S. Pat. No. 5,680,253 discloses a light beam scanning apparatus comprising a first and second diffraction grating plate which provides a high resolution scanning apparatus utilizing mass producible holograms instead of utilizing auxiliary optical systems. An optical system for a large depth of field barcode scanner was suggested in U.S. Pat. No. 4,978,860 to Bayley et al. The working range of this device is determined not by the lens aperture, but by the dimension and orientation of the detector array using a modified Scheimpflug arrangement.
Nevertheless, the existing prior art devices do not fully solve the technical problems required for imaging systems to be used for distance estimation and collision warning for vehicles.
The invention provides super-resolving devices and their applications. More particularly, it provides geometrical super-resolution, i.e. resolution that is better than the resolution defined by the geometry of the detector array. In addition, the disclosed technology also provides apparatus to extend the depth of field of the system.
It is a primary object of some aspects of the present invention to improve imaging systems in the fields of distance estimation and collision warning for vehicles.
It is another object of some aspects of the present invention to provide apparatus for estimating distance that affords wide depth of focus and a large horizontal field of view.
It is yet another object of some aspects of the present invention to improve the speed of distance estimation in collision warning systems.
These and other objects of the present invention are attained by imaging apparatus in which special optical elements placed on the imaging lens and the detector array allow the conversion of degrees of freedom in one axis of a field of view to a larger degree of freedom in another axis in order to obtain a high resolution image with a wide depth of focus and large field of view suitable for performing distance estimation on an object within the field of view.
The invention employs a numerical algorithm which is simple and can be rapidly performed to yield the desired result.
According to some aspects of the invention a detector array with a specially designed optical element is directed toward a large volume of space in which a target may be found. In certain applications, such as vehicular warning systems, the vertical field of view required is significantly smaller than the actual vertical field of view of the apparatus. The special optical element converts the unneeded vertical pixels (degrees of freedom) into an increased depth of focus resolution. The apparatus provides a view comprising several horizontal xe2x80x9cstripesxe2x80x9d wherein each stripe represents a planar object and corresponds to an optical element which has both a unique focal length and thus a unique depth of focus region. In a preferred embodiment of the invention a distance evaluation is performed by evaluating the amount of misfocus of each of the various stripes.
In an alternate preferred embodiment of the invention the distance estimation is performed by employing two cameras, each having similar structure to that of the first embodiment. A triangulation calculation is performed in order to extract the distance of the object seen by both cameras.
In another aspect of the invention, a diffractive optical element that is attached to the aperture of the system further improves the geometrical resolution of the imaged scene by converting vertical degrees of freedom into horizontal resolution. A system including a camera employing the apparatus according to the invention provides high resolution performance both in the vertical and in the depth axes together enabling determination of the desired distance estimation.
In yet another aspect of the invention an optimal filtering approach is applied in order to perform the recognition of the high-resolution image.
The invention provides a method of imaging, which is performed by tilting an image plane with respect to an object plane, defining pixel units on the image plane, and defining a field of view on the object plane, wherein a vertical dimension of the field of view differs from a horizontal dimension thereof. A plurality of images of the field of view are formed on the image plane, the images being spatially interrelated by subpixel shifts. The formed images are then super-resolved into an enhanced image.
According to an aspect of the invention the step of super-resolving is geometrically performed by transforming the images according to a Gabor transform.
According to another aspect of the invention the step of super-resolving also has applying a spatial mask to the image plane.
According to still another aspect of the invention the images are oriented according to the larger of the horizontal dimension and the vertical dimension.
In a further aspect of the invention there is a further step of deconvolving out-of-focus point spread functions of the images, preferably using a Wiener filter.
According to an aspect of the invention the images are formed by time multiplexing.
According to another aspect of the invention the images are formed by wavelength multiplexing.
According to yet another aspect of the invention the images are formed by diffractive gratings.
According to still another aspect of the invention the images are transformed according to a Mellin transform, and time-to-impact analysis is performed on the transformed images.
In a further aspect of the invention the images are transformed such that their impulse responses are diagonally displaced from one another on the image plane with respect to horizontal and vertical axes of the images defined thereon.
The invention provides a method of imaging which is performed by simultaneously viewing a plurality of elongated fields of view on a target, wherein a focus of each field of view has a different optical displacement from an imaging surface, diffracting beams of radiant energy that travel along a path extending between the fields of view and the imaging surface to form elongated replicas of each elongated field of view, the replicas being mutually shifted in a direction of elongation thereof, and being mutually displaced in a direction that is substantially transverse to the direction of elongation. The method is further performed by combining the replicas of each field of view into corresponding enhanced images, each enhanced image having a higher resolution than the resolutions of its associated replicas.
Preferably the image plane is tilted with respect to planes of the fields of view, and the method further comprises determining defocus of the fields of view.
According to another aspect of the invention the method further comprises performing a Mellin transform on an image formed from all of the replicas.
In a further aspect of the invention the method comprises conducting the beams through a spatial mask that is constructed such that zero values of a Fourier transform of a function representing spatial responsivity of pixels on the image plane are eliminated.
The invention provides a method of imaging, which is performed by tilting an image plane with respect to first and second object planes intersecting a target, wherein a displacement of the second object plane from the image plane is greater than the displacement of the first object plane from the image plane. The method includes defining pixel units on the image plane, defining a first field of view on the first object plane, and defining a second field of view on the second object plane, wherein vertical dimensions of the first and second fields of view differ from respective horizontal dimensions thereof. The method further comprises forming a plurality of first images of the first field of view on the image plane, the first images being spatially interrelated by subpixel shifts, and simultaneously forming a plurality of second images of the second field of view on the image plane, the second images being spatially interrelated by subpixel shifts. The method further comprises super-resolving the first images into a first enhanced image, and super-resolving the second images into a second enhanced image, and comparing a defocus magnitude of the first field of view with a defocus magnitude of the second field of view.
Preferably the first field of view is obtained through a first region of a lens, and the second field of view is obtained through a second region of the lens.
According to an aspect of the invention the step of super-resolving is geometrically performed by transforming the first images and the second images according to a Gabor transform.
According to another aspect of the invention the step of super-resolving further comprises applying a spatial mask to the image plane.
According to still another aspect of the invention the plurality of first images and the plurality of the second images are oriented according to the larger of the horizontal dimension and the vertical dimension.
According to an aspect of the invention the plurality of images are formed by spatially transforming coordinates to establish a panoramic field of view, in which has a horizontal field differs in magnitude from the vertical field of view.
In a further aspect of the invention the method further comprises determining a displacement between the image plane and the target, responsive to the step of comparing defocus magnitudes.
In another aspect of the invention the first images and the second images are spaced apart, and a position of the target is determined with respect to the first and second images by triangulation.
The invention provides an imaging apparatus, including a lens which has a field of view that includes an object plane, wherein a horizontal dimension of the field of view differs from a vertical dimension thereof. A detector of radiant energy has an image formed thereon by the lens, wherein the detector has a plurality of pixel elements, and a spatial mask is disposed proximate the detector. The mask has a plurality of subpixel apertures formed therein. The apparatus further includes a diffractive element disposed proximate the lens, wherein a diffracted light beam forms the image on the detector, and the image comprises a plurality of focused replicas of the field of view, the focused replicas being offset from one another by subpixel shifts, and a prism is disposed between the lens and the detector for refracting radiant energy passing therethrough.
According to an aspect of the invention the spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of the pixel elements.
According to another aspect of the invention the diffractive element is a one dimensional zone plate strip.
According to yet another aspect of the invention the replicas are mutually shifted in a direction of elongation of the Barfield of view, and are spaced apart from one another in a direction that is substantially transverse to the direction of elongation.
According to still another aspect of the invention the apparatus further includes a signal processing unit which is responsive to the detector, and is programmed to combine the replicas into a single image which has a greater resolution than the repilicas.
In a further aspect of the invention the signal processing unit performs a Gabor transform on a signal output of the detector.
In another aspect of the invention the signal processing unit performs a Mellin transform on a signal output of the detector in conjunction with time-to-impact analysis.
According to an aspect of the invention there are first and second lenses spaced apart from one another.
According to another aspect of the invention the signal processing unit is programmed to determine a displacement by triangulation of a target using a first image thereof obtained from the first lens and a second image thereof obtained from the second lens.
According to still another aspect of the invention an aperture of the lens is divided into a first region and a second region, wherein the signal processing unit is programmed to determine a displacement by triangulation of a target, using a first image thereof obtained from the first region and a second image thereof obtained from the second region of the lens.
The invention provides an imaging apparatus, including a plurality of lenses each having a corresponding field of view that includes an object plane, wherein a horizontal dimension of the field of view differs from a vertical dimension thereof, the lenses having different focal lengths. The apparatus further includes a detector of radiant energy having a plurality of corresponding images formed thereon by the lenses, a spatial mask disposed proximate the detector, the mask having a plurality of subpixel apertures formed therein. The apparatus further includes a diffractive element disposed proximate each lens, wherein each image on the detector has a plurality of focused replicas of the corresponding field of view, the focus replicas are offset from one another by subpixel shifts, and a prism is disposed between the lens and the detector for refracting radiant energy passing therethrough onto the detector.
In a further aspect of the invention the prism has a plurality of prisms, each prism refracting radiant energy that is received from a corresponding lens.
Preferably the spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of the pixel elements.
According to an aspect of the invention the diffractive element is a one dimensional zone plate strip.
According to another aspect of the invention the replicas are mutually shifted in a direction of elongation of the corresponding field of view, and are spaced apart from one another in a direction that is substantially transverse to the direction of elongation.
According to yet another aspect of the invention the apparatus further includes a signal processing unit which is responsive to the detector, and is programmed to combine the replicas of each the field of view into a corresponding single image which has a greater resolution than the replicas.
According to another aspect of the invention the lens includes means for spatially transforming the plurality of focused replicas to establish a panoramic field of view, in which has a horizontal field differs in magnitude from the vertical field of view.
According to still another aspect of the invention the signal processing unit performs a Gabor transform on a signal output of the detector.
In a further aspect of the invention the signal processing unit performs a Mellin transform on a signal output of the detector.
Preferably the signal processing unit deconvolves out-of-focus point spread functions of the images.
According to an aspect of the invention the signal processing unit employs a Wiener filter for deconvolving the point spread functions.